Businesses and other enterprises and entities are increasingly utilizing computer networks to manage their business and other activities. Such networks provide numerous benefits including extended access to localized resources, rapid sharing of information and so forth. Thus, a number of applications and services are now designed to run over or utilize network connections as part of their normal function. As the networks used by businesses and others become larger and more highly populated, the issue of scalability must be considered. For example, a distributed system that works well with 10 computers may fail catastrophically when used with 1000 computers. Such failures are difficult to predict generally, and thus scalability testing is typically performed to verify the proper operation of products intended for large scale application.
Typically, only one copy of a given application (e.g. a client portion of a distributed service) can be run on each client machine, so special techniques are typically used to perform scalability testing to verify operation over a vast network. Techniques that have been used in the past to perform scalability testing include (1) simply using the system under test on the required number of machines and observing the results, (2) simulation, (3) emulation, and (4) alteration of the product under test to allow multiple copies to run on a given machine. Each of these techniques, however, has significant deficiencies.
With respect to the first technique mentioned above, the use of a collection of machines to directly test scalability with respect to a network of the same number of machines is not feasible for large networks. In particular, network sizes can be so large that the cost of securing and setting up the proper number of machines is prohibitive for the tester. With respect to simulation, this technique is also not feasible for very large networks since, among other problems, the load on the system resources of the testing machines becomes quite severe. With respect to emulation, typically the software that is run during the test is an altered form of the software for which scalability testing is desired. Given this, several drawbacks are apparent, including the need to independently develop and test the new version. In addition the test is an indirect test at best since it does not test the actual software of interest.
Finally, the modification of the software of interest to allow multiple copies to run simultaneously on a given machine has a number of drawbacks, including some of those mentioned above. The use of such modified applications still taxes the system resources as with simulation, requires resources for creation of the modified version, and in addition does not provide a test of the actual product of interest since it tests a specialized version.
Thus, although scalability of software products to large networks is desirable and in many cases necessary, existing methods of testing such scalability are unsatisfactory. A method of scalability testing is needed whereby a software product or system can itself be tested for scalability without encountering the deficiencies found in prior testing systems and techniques.